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Mr. Chairman and Members of the Subcommittee: 


I am pleased to be here today to discuss our work, requested by this 
Subcommittee, on the National Aeronautics and Space Administration’s 
(NASA) X-33 and X-34 programs. As you know, the purpose of these 
efforts was to significantly reduce the cost of access to space by 
partnering with private industry to develop and demonstrate technologies 
needed for future reusable launch vehicles reaching orbit in one stage 
(single-stage-to-orbit). In essence, these are vehicles whose components— 
either all or in part—can be utilized on subsequent flights. Both programs 
were recently terminated because of significant cost increases caused by 
problems developing the necessary technologies and flight demonstration 
vehicles. NASA is now focusing instead on its new Space Launch Initiative. 
This is a broader effort to develop the next generation of reusable launch 
vehicles, referred to as the Second Generation Reusable Launch Vehicle 
Program (2 nd Generation Program). Today, I will discuss the primary 
factors that contributed to the difficulties experienced by the X-33 and X- 
34 programs and the steps needed to avoid repeating those problems 
within the 2 nd Generation Program. 

In brief, the X-33 and X-34 programs experienced difficulties achieving 
their goals primarily because NASA did not develop realistic cost 
estimates, timely acquisition and risk management plans, and adequate 
and realistic performance goals. In particular, neither program fully 
assessed the costs associated with developing new, unproven 
technologies; provided for the financial reserves needed to deal with 
technical risks and accommodate normal development delays; developed 
plans to quantify and mitigate the risks to NASA; or established 
performance targets showing a clear path leading to an operational 
reusable launch vehicle. Underlying these difficulties were problems with 
the agreements and contracts that established the relationship between 
NASA and its industiy partners and eventual erosion of commercial 
prospects for the development of new reusable launch vehicles. 

Currently, NASA is in the process of taking steps in the 2 nd Generation 
Program to help avoid problems like those encountered in the X-33 and X- 
34 programs. While it is too early to tell if these measures will be 
sufficient, our review of the two programs has shown that three critical 
areas need to be addressed. First, the technical complexity involved 
requires that realistic cost estimates and risk mitigation plans are 
developed and the projects are funded accordingly. Second, because the 
2 nd Generation Program will involve numerous interrelated, complex 
efforts to develop technology, NASA needs to ensure that all these efforts 


Page 1 


GAO-01-826T 



move forward in a coordinated manner and that open communication is 
fostered at all levels. Third, performance measures need to be 
implemented and periodically validated to ensure that the rationale for 
developing specific technology applications merits continued support. 


Background 


The X-33 and X-34 programs were part of an effort that began in 1994— 
known as the Reusable Launch Vehicle Technology/Demonstrator 
Program (Reusable Launch Vehicle Program)—to pave the way to full- 
scale, commercially-developed, reusable launch vehicles reaching orbit in 
one stage. In embarking on the Reusable Launch Vehicle Program, NASA 
sought to significantly reduce the cost of developing, producing and 
operating launch vehicles. NASA’s goal was to reduce payload launch 
costs from $10,000 per pound on the space shuttle to $1,000 per pound. It 
planned to do so, in part, by finding “new ways of doing business” such as 
using innovative design methods, streamli n ed acquisition procedures, and 
creating industry-led partnerships with cost sharing to manage the 
development of advanced technology demonstration vehicles. The 
vehicles were seen as the “stepping stones” in what NASA described as an 
incremental flight demonstration program. The strategy was to force 
technologies from the laboratory into the operating environment. 

The X-34 Project started in 1995 as a cooperative agreement between 
NASA and Orbital Sciences Corporation u (Orbital). The project was to 
demonstrate streamlined management and procurement, industry cost 
sharing and lead management, and the economics of reusability. However, 
the industry team withdrew from the agreement in less than 1 year, for a 
number of reasons including changes in the projected profitability of the 
venture. NASA subsequently started a new X-34 program with a smaller 
vehicle design. It was intended only as a flight demonstration vehicle to 
test some of the key features of reusable launch vehicle operations, such 
as quick turn-around times between launches. 

Under the new program, NASA again selected Orbital as its contractor in 
August 1996, awarding it a fixed price, $49.5 million contract. Under the 
new contract, Orbital was given lead responsibility for vehicle design, 
fabrication, integration, and initial flight testing for powered flight of the 
X-34 test vehicle. The contract also provided for two options, which were 


Rockwell International was an industry partner with Orbital Science Corporation in the 
development of the commercial reusable vehicle. 
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later exercised, totaling about $17 million for 25 additional experimental 
flights and, according to a project official, other tasks, including defining 
how the flight tests would be undertaken. Under the new effort, NASA’s 
Marshall Space Flight Center was to develop the engine for the X-34 as 
part of its Low Cost Booster Technology Project. The initial budget for this 
development was about $18.9 million. 

In July 1996, NASA and Lockheed Martin Corporation and its industry 
part.ners4mt.ered into a cooperative agreement for the design, 
development, and flight-testing of the X-33. ^The X-33 was to be an 
unmanned technology demonstrator. It would take off vertically like a 
rocket, reaching an altitude of up to 60 miles and speeds to about Mach 13 
(13 times the speed of sound), and land horizontally like an airplane. The 
X-33 would flight test a range of technologies needed for future launch 
vehicles, such as thermal protection systems, advanced engine design and 
lightweight fuel tanks made of composite materials. The vehicle would not 
actually achieve orbit, but based on the results of demonstrating the new 
technologies, NASA envisioned being in a better position to make a 
decision on the feasibility and affordability of building a full-scale system. 
Under the initial terms of the cooperative agreement, NASA’s contribution 
was fixed at $912.4 million and its industry partners’ initial contribution 
was $211.6 million. In view of the potential commercial viability of the 
launch vehicle and its technologies, the industry partners also agreed to 
finance any additional costs. During a test in November 1999, one of the 
fuel tanks failed due to separation of the composite surface. Following the 
investigation, NASA and Lockheed Martin agreed to replace the composite 
tanks with aluminum tanks. 

In February 2001, NASA announced it would not provide any additional 
funding for the X-33 or X-34 programs under its new Space Launch 
Initiative. The Space Launch Initiative is intended to be a more 
comprehensive, long-range plan to reduce high payload launch costs. 
NASA’s goal is still to reduce payload launch cost to $1,000 per pound to 
low Earth orbit but it is not limited to single-stage-to-orbit concepts. 


2 Lockheed Martin made agreements with Allied Signal Aerospace, B.F. Goodrich 
Aerospace, Boeing-Rocketdyne Division, and Sverdrup Corporation for the X-33 Program. 

3 NASA’s use of a cooperative agreement allowed the industry partners, and NASA, to 
withdraw from the agreement without penalty at any time. 

4 We reported on X-33 costs in Space Transportation: Status of the X-33 Reusable Launch 
Vehicle Program (GAO/NSIAD-99-176, Aug. 11, 1999). 
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Specifically, the 2 nd Generation Program’s objective is to substantially 
reduce the technical, programmatic, and business risks associated with 
developing reusable space transportation systems that are safe, reliable 
and affordable. 

NASA has budgeted about $900 million for the SLI initial effort and, in May 
2001, it awarded initial contracts to 22 large and small companies for 
space transportation system design requirements, technology risk 
reduction, and flight demonstration. In subsequent procurements in mid¬ 
fiscal year 2003, NASA plans to select at least two competing reusable 
launch system designs. The following 2.5 to 3.5 years (through fiscal years 
2005 or 2006) will be spent finalizing the preliminary designs of the 
selected space transportation systems, and maturing the specific 
technologies associated with those high-risk, high-priority items needed to 
develop the selected launch systems. 


Factors Contributing 
to X-33 and X-34 
Program Difficulties 


Undertaking ambitious, technically challenging efforts like the X-33 and X- 
34 programs—which involve multiple contractors and technologies that 
have yet to be developed and proven—requires careful oversight and 
management. Importantly, accurate and reliable cost estimates need to be 
developed, technical and program risks need to be anticipated and 
mitigated, sound configuration controls need to be in place, and 
performance needs to be closely monitored. Such undertakings also 
require a high level of communication and coordination. Not carefully 
implementing such project management tools and activities is a recipe for 
failure. Without realistically estimating costs and risks, and providing the 
reserves needed to mitigate those risks, management may not be in a 
position to effectively deal with the technical problems that cutting-edge 
projects invariably face. 


In fact, we found that NASA did not successfully implement and adhere to 
a number of critical project management tools and activities. Specifically: 

• NASA did not develop realistic cost estimates in the early stages of the X- 
33 program. From its inception, NASA officials considered the program to 
be high risk, with a success-oriented schedule that did not allow for major 
delays. Nevertheless, in-September 1999, NASA’s Office of the Inspector 
General (OIG) reported^that NASA’s cost estimate did not include a risk 


5 Audit Report: X-33 Cost Estimating Processes, NASA Office of Inspector General, IG-99- 
052, 
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analysis to quantify technical and schedule uncertainties. Instead, the cost 
estimate assumed that needed technology would be available on schedule 
and as planned. According to the OIG, a risk analysis would have alerted 
NASA decision-makers to the probability of cost overruns in the program. 
Since NASA’s contribution to the program was fixed—with Lockheed 
Martin and its industry partners responsible for costs exceeding the initial 
$1.1 billion—X-33 program management concluded that there was no risk 
of additional government financial contributions due to cost overruns. 
They also believed that the projected growth in the launch market and the 
advantages of a commercial reusable launch vehicle would provide the 
necessary incentive to sustain industry contributions. 

• NASA did not prepare risk management plans for both the X-33 and X-34 
programs until several years after the projects were implemented. Risk 
management plans identify, assess, and document risks associated with 
cost, resource, schedule, and technical aspects of a project and determine 
the procedures that will be used to manage those risks. In doing so, they 
help ensure that a system will meet performance requirements and be 
delivered on schedule and within budget. A risk management plan for the 
X-34 was not developed until the program was restructured in June 2000. 
Although Lockheed Martin developed a plan to manage technical risks as 
part of its 1996 cooperative agreement for the X-33, NASA did not develop 
its own risk management plan for unique NASA risks until February 2000. 
The NASA Administrator and the NASA Advisory Council have both 
commented on the need for risk plans when NASA users partnering 
arrangements such as a cooperative agreement. Furthermore, we found 
that NASA’s risk mitigation plan for the X-33 program provided no 
mechanisms for ensuring the completion of the program if significant cost 
growth occurred and/or the business case motivating industry 
participation weakened substantially. 

• Against its own policy, NASA did not prepare program commitment 
agreements or program plans at the onset for either program. The 
commitment agreement lays out the program’s technical, schedule, and 
cost commitments, and overall acquisition strategy. The program plan 
addresses these issues as well but also defines the effort’s management 
structure as well as program resources, data management, risk 
management, test and verification, and planned program reviews. Such 
plans would help NASA to define realistic time frames, identify 


Sept. 24, 1999. 


Page 5 


GAO-01-826T 



responsibility for key tasks and deliverables and provide a yardstick by 
which to measure the progress of the effort. 


• According to the OIG, NASA did not complete a configuration 
management plan for the X-33 until May 1998—about 2 years after NASA 
awarded the cooperative agreement and Lockheed Martin began the 
design and development of a flight demonstration vehicle. Configuration 
management plans define the process to be used for defining the 
functional and physical characteristics of a product and systematically 
controlling changes in the design. As such, they enable organizations to 
establish and maintain the integrity of a product throughout its lifecycle 
and prevent the production and use of inconsistent product versions. By 
the time the plan was implemented, hardware for the demonstration 
vehicle was already being fabricated. 

• Communications and coordination were not effectively facilitated. In a 
report following the failure of the X-33’s composite fuel tank, the 
investigation team reported that the design of the tank required high levels 
of communication, and that such communication did not occur in this 
case. A NASA official told us that some NASA and Lockheed personnel, 
who had experience with composite materials and the phenomena 
identified as one of the probable causes for the tank’s failure, expressed 
concerns about the tank design. However, because of the industry-led 
nature of the cooperative agreement, Lockheed Martin was not required to 
react to such concerns and did not request additional assistance from 
NASA. 

• The Government Performance and Results Act of 1993 requires federal 
agencies to prepare annual performance plans to establish measurable 
objectives and performance targets for major programs. Doing so enables 
agencies to gauge the progress of programs like the X-33 and X-34 and in 
turn to take quick action when performance goals are not being met. For 
example, we reported in August 1999 that NASA’s Fiscal Year 2000 
Performance Plan did not include performance targets that established a 
clear path leading to a reusable launch vehicle and recommended such 
targets be established. 


e Final Report of the X-33 Liquid Hydrogen Tank Test Investigation Team. National 
Aeronautics and Space Administration, George C. Marshall Space Flight Center, Huntsville, 
Alabama 35812, May 

2000 . 
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Without relying on these important project management tools up front, 
NASA encountered numerous problems on both the X-33 and X-34 
programs. Compounding these difficulties was a decrease in the projected 
commercial launch market, which in turn lessened the incentive of NASA’s 
X-33 industry partners to continue their investments. 

In particular, technical problems in developing the X-33’s composite fuel 
tanks, aerospike engines, heat shield, and avionics system resulted in 
significant schedule delays and cost overruns. After two program reviews 
in 1998 and 1999, the industry partners added a total of $145.6 million to 
the cooperative agreement to pay for cost overruns and establish a reserve 
to deal with future technical problems and schedule delays. However, 
NASA officials stated that they did not independently develop their own 
cost estimates for these program events to determine whether the 
additional funds provided by industry would be sufficient to complete the 
program. Also, these technical problems resulted in the planned first flight 
being delayed until October 2003, about 4.5 years after the original March 
1999 first flight date. 

After the composite fuel tank failed during testing in November 1999, 
according to NASA officials, Lockheed Martin opted not to go forward 
with the X-33 Program without additional NASA financial support. 
Lockheed Martin initially proposed adding $95 million of its own funds to 
develop a new aluminum tank for the hydrogen fuel, but also requested 
about $200 million from NASA to help complete the program. Such 
contributions would have increased the value of the cooperative 
agreement to about $1.6 billion or about 45 percent (about $500 million) 
more than the $1.1 billion initial cooperative agreement funding. NASA did 
not have the reserves available to cover such an increase. The agency did, 
however, allow Lockheed Martin to compete, in its 2 nd Generation Program 
solicitation for the additional funds Lockheed Martin believed it needed to 
complete the program. 

Similarly, NASA started the X-34 Project, and the related NASA engine 
development project, with limited government funding, an accelerated 
development schedule, and insufficient reserves to reduce development 
risks and ensure a successful test program. Based on a NASA X-34 
restructure plan in June 2000, we estimate that NASA’s total funding 
requirements for the X-34 would have increased to about $348 million—a 
307-percent ($263 million) increase from the estimated $86 million 
budgeted for the vehicle and engine development projects in 1996. Also, 
since 1996, the projected first powered flight had slipped about 4 years 
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from September 1998 to October 2002 due to the cumulative effect of 
added risk mitigation tasks, vehicle and engine development problems, 
and testing delays. 

Most of the cost increase (about $213 million) was for NASA-directed risk 
mitigation tasks initiated after both projects started. For example, in 
response to several project technical reviews and internal assessments of 
other NASA programs, the agency developed a restructure plan for the X- 
34 project in June 2000. This plan included consolidating the vehicle and 
engine projects under one NASA manager. The project would be managed 
with the NASA project manager having the final decision-making 
authority; Orbital would be relegated to a more traditional subordinate 
contractor role. Under the plan, the contract with Orbital would also be 
rescoped to include only unpowered flights; Orbital would have to 
compete for 2 nd Generation Program funding for all the powered flight 
tests. The plan’s additional risk mitigation activities would have increased 
the X-34 project’s funding requirements by an additional $139 million, 
which included about $45 million for additional engine testing and 
hardware; $33 million for an avionics redesign; $42 million for additional 
project management support and personnel; and $18 million to create a 
contingency reserve for future risk mitigation efforts. 


Incorporating Lessons 
Learned in 2 nd 
Generation Program 


NASA is revising its acquisition and management approach for the 2 nd 
Generation Program. Projects funded under the program will be NASA-led 
rather than industry-led. NASA also plans to increase the level of insight 
into the program’s projects, for example, by providing more formal 
reviews and varying levels of project documentation from contractors 
depending on the risk involved and the contract value. NASA also required 
that all proposals submitted in response to its research announcement be 
accompanied by certifiable cost and pricing data. Finally, NASA 
discouraged the use of cooperative agreements since these agreements did 
not prove to be effective contract vehicles for research and development 
efforts where large investments are required. 


While it is too early to tell if the agency measures aimed at avoiding the 
problems experienced in the X-33 and X-34 programs will be sufficient, 


'These assessments include NASA internal reports on failures in the Mars Program, Shuttle 
wiring problems and an assessment of NASA’s approach to executing “Faster, Better, 
Cheaper.” 
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these experiences show that three critical areas need to be addressed. 
These relate to (1) adequate project funding and cost risk provisions, (2) 
the effective and efficient coordination and communication required by 
many individual but related efforts, and (3) periodically revalidating 
underlying assumptions by measuring progress toward achieving a new 
safe, affordable space transportation system that meets NASA’s 
requirements. 

First, the technical complexity of the 2 nd Generation Program requires that 
NASA develop realistic cost estimates and risk mitigation plans, and 
accordingly set aside enough funds to cover the program’s many projects. 
NASA plans to invest substantially more funds in the 2 nd Generation 
Program than it did in the previous Reusable Launch Vehicle Program, and 
plans to provide reserves for mitigating program risk. For example, the 
agency plans to spend about $3.3 billion over 6 years to define system 
requirements for competing space transportation systems and related risk 
reduction activities. Most of this amount, about $3.1 billion, is for risk- 
reduction activities, such as the development of new lightweight 
composite structures, durable thermal protection systems, and new high 
performance engine components. 

NASA officials told us that an important way they plan to mitigate risk is 
by ensuring adequate management reserves in the 15- to 20-percent range, 
or higher if needed. They also acknowledged the need for adequate 
program cost estimates on which to base reserve requirements. However, 
we are still concerned about the timely preparation of cost estimates. The 
2 nd Generation deputy program manager stated that, based on the scope of 
the first contracts awarded, the program office planned to update their 
cost estimate this summer before NASA conducted a separate, 
independent technical review and cost estimate in September 2001. Thus, 
neither of these important analyses were completed prior to the first 
contract awards. We believe that until the program office completes it own 
updated cost estimate and NASA conducts an independent cost and 
technical review, a credible estimate of total program costs and the 
adequacy of planned reserves will not be available. Also, NASA is still in 
the process of developing the documentation required for the program, 
including a risk mitigation plan. NASA policy requires that key program 
documentation be finalized and approved prior to implementing a 
program. 

Second, NASA will face coordination and communication challenges in 
managing the 2 nd Generation Program. As noted earlier, NASA recently 
awarded initial contracts for systems engineering and various risk 
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reduction efforts to 22 different contractors. Yet to successfully carry out 
the program NASA must, early on, .have coordinated insight into all of the 
space transportation architecturesMbeing proposed by these contractors 
and their related risk reduction activities. Clearly, this will be a significant 
challenge. The contractors proposing overall architecture designs must be 
aware of all the related risk reduction development activities affecting 
their respective designs. It may also prove difficult for contractors 
proposing space transportation system designs to coordinate work with 
other contractors without a prime contractor to subcontractor 
relationship. NASA’s own Aerospace Technology Advisory Committee, 
made up of outside experts, has also expressed serious concerns about the 
difficulty of integrating these efforts effectively. 

The need for improvement in coordination and communications in all 
NASA programs has been noted in the past and is not unique to the X-33 
and X-34 programs. We and other NASA investigative teams have found 
and noted similar problems with other NASA programs such as the 
Propulsion Module for the International Space Station, and several other 
projects including the two failed Mars missions. NASA’s Space Launch 
Initiative Program would benefit from lessons learned from past mishaps. 
At the request of the House Science Committee, we are undertaking a 
review of NASA’s lessons learned process and procedures. The principal 
objectives of this review are to determine (1) how NASA captures and 
disseminates lessons learned and (2) if NASA is effectively applying 
lessons learned toward current programs and projects. We will report the 
results of our evaluation in December of this year. 

The third challenge is establishing performance measures that can 
accurately gauge the progress being made by NASA and its contractors. 
NASA officials told us that they plan to periodically reassess the 
assumptions underlying key program objectives to ensure that the 
rationale for developing specific technology applications merits continued 
support. They also told us that they were in the process of establishing 
such metrics to measure performance. Ensuring that the results from the 
2 nd Generation Program will support a future decision to develop reusable 
launch vehicles also deserves attention in NASA’s annual Performance 
Plan. The plan would be strengthened by recognizing the importance of 


8 A space transportation architecture is defined as an Earth-to-orbit launch vehicle, on-orbit 
transfer vehicles and upper stages, mission planning, ground and flight operations, and 
support infrastructure. 
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clearly defined indicators which demonstrate that NASA is (1) on a path 
leading to an operational reusable launch vehicle and (2) making progress 
toward its objective of significantly reducing launch costs, and increasing 
safety and reliability compared to existing systems. Affected NASA 
Enterprise and Center performance plans would also be strengthened with 
the development of related metrics. 


Mr. Chairman, this concludes my statement. I would be happy to answer 
any questions you or other Members of the Subcommittee may have. 


Objectives, Scope, 
And Methodology 


We interviewed officials at NASA headquarters in Washington D.C., 

NASA’s Marshall Space Flight Center, Huntsville, Alabama, and at the 
NASA X-33 program office at Palmdale, California to (1) determine the 
primary program management factors that contributed to the difficulties 
experienced in the X-33 and X-34 programs, and (2) to identify steps that 
need to be taken to avoid repeating those problems within the Space 
Launch Initiative framework. We also talked to representatives of NASA’s 
Independent Program Assessment Office located at the Langley Research 
Center, Hampton, Virginia and the OIG located at NASA headquarters and 
Marshall Space Flight Center. At these various locations we obtained and 
analyzed key program, contractual and procurement documentation for 
the X-33, X-34 and 2 nd Generation programs. Further, we reviewed reports 
issued by the NASA’s OIG and Independent Program Assessment Office 
pertaining to the management and execution of the X-33 and X-34 
programs, and NASA Advisory Council minutes regarding NASA’s efforts 
to develop reusable launch vehicles. In addition, we reviewed other NASA 
internal reports documenting management issues associated with program 
formulation and implementation of other NASA programs. We also 
reviewed applicable NASA policy regarding how NASA expects its 
programs and projects to be implemented and managed. 

We conducted our review from August 2000 to June 2001 in accordance 
with generally accepted government auditing standards. 
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For information about this testimony, please contact Allen Li at (202) 512- 
4841. Contributors to this testimony included Jerry Herley, Noel Lance, 
and Carlos Garcia. 
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